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ABSTRACT 

In 1976, the Montgomery County Public Schools began 
the development of a kindergarten through grade 8 Instructional 
Program in Reading/Language Arts UPR/LA) for teaching the English 
language arts. Goals of the effort were to improve reading and 
listening comprehension instruction and to improve program 
consistency across grades, schools, and areas. The first full year of 
data collection for the reading program was 1982-83. During that year 
the focus of study centered on descriptive information about the 
characteristics of reading instruction and problems in the 
implementation process in the study schools. A sample of 20 
elementary schools was selected for the reading study and stratified 
on the basis of achievement level and length of time the school had 
been exposed to I PR/LA. Study findings indicated that while some 
teachers were able to implement the revised curriculum in a number of 
areas, somt serious problems still remained. Further, the program was 
not implemented for students at all achievement levels. In addition, 
no consistent criteria were in use system-wide to monitor program 
implementation. Neither the training nor the materials provided to 
schools to support the implementation process were perceived as 
adequate. Finally, the irole of the reading teacher appeared to have 
changed, which created role conflicts for some school-based reading 
teachers. A technical appendix includes data collection procedures 
and instruments used in classroom observation. (HOD) 
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EXECUTIVE SUMMARY 
READING STUDY: FIRST-YEAR REPORT 



BACKGROUND 

In 1976, MCPS began development of the K-6 Instructional Program in 
Reading/Language Arts (IPR/LA) for improving the teaching of the English 
Language Arts. The objectives for the new program were approved by the Board 
of Education in June of 1981 as the mandated Program of Studies for 
reading/listening for all students in Trades K-8. The IPR/LA Includes the 
mandated objectives and a variety of recommended support materials (e.g., 
teachers 9 guides and tests). Materials for the narration component have 
been completed, while those for other forms of discourse (exposition, 
persuasion) are in various stages of implementation. The program Is 
currently being implemented in schools under the direction of area offices. 
The Reading Study is a multlyear evaluation of the narratiou component of 
IPR/LA. Its purpose is to judge systematically the degree to which schools 
are actually using the new curriculum and its Impact on reading instruction 
and student achievement. 

A sample of 20 MCPS elementary schools was selected for the Reading Study, 
stratified on the basis of achievement level (above or below the county mean 
in reading) and length of time the school had been exposed to IPR/LA. 
Schools from each area were included (see Exhibit 1 for a list of study 
schools by strata). Two schools in the sample had not officially begun 
Implementation and were initially included to provide some baseline 
information on classroom characteristics prior to program implementation. 
Data collection revealed, however, that many teachers at these schools had 
been exposed to the program at other schools prior to the study. Thus, 
their classroom practices could not be considered descriptive of those prior 
to implementation, and no baseline data are reported in the study. 

The 1982-83 school year was the first full year of data collection. 
Respondents Included area staff, principals, school specialists, and 
classroom teachers. Within each sample school, two first and two fourth 
grade classrooms were randomly selected for the first year of the study for 
classroom observations and student data collection. The approximately 1800 
sample students in these classes will be followed for the duration of the 
study. Analyses of first year data focused on descriptive information about 
the characteristics of reading instruction and implementation problems 
systemwlde. 

POLICY ISSUES 

This report focuses on several policy Issues that can be addressed using the 
data already collected. The three major Issues examined in this report are 
the following: 

o Is IPR/LA being Implemented for nil students? 
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EXHIBIT 1 






Sample Schools for the Reading Study 




Years Implementing 
Curriculum 


Achievement Level 
Below County Above County 
Mean Hean 


Total 


Since 1981-82 


Broad Acres 
Maryvale 
Rolling Terrace 


Potomac 

Fallsmead 

Sherwood 


6 


From 1980-81 


*Beall 
Fieldr Road 
Viers Mill 


Seven Locks 
Cedar Grove 
Greenwood 


6 


Prior to 1980 


Poclesville 

Clarksburg 

Gletwllan 


. Bannockburn 
Lay tonsville 
Kemp Mill 


6 


Nonimplementlng schools 




Be the s da 
Farmland 


2 
20 


*Beall consolidated with a school in this category of implementation. 
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Are the supports, training, and materials necessary for 
implementation being provided to school personnel? 

What roles are played by the different administrative levels in 
monitoring and managing the implementation process? 



FINDINGS 

PROGRAM IMPLEMENTATION 

IPR/LA provides an approach to teaching reading/listening comprehension that 
encompasses different instructional strategies as well as differential use 
of materials such as basal readers and tradebooks. Study findings indicate 
that while some teachers were able to implement the revised curriculum in a 
number of areas, some serious problems still remained. Further, the program 
was not being implemented for students at all achievement levels; low- 
achieving students in particular often received what might be characterized 
as a "different program." These variations were not related to the amount 
of time a school had officially been implementing the program. Specific 
findings were the following: 

o Instructional strategies recommended by the curriculum guides 
(setting a purpose for reading, using students' background 
knowledge, and open-ended questioning) were in fairly widespread 
use in study schools. Teachers were observed using these 
strategies with 53 to 77 percent of the groups in reading 
instruction. The degree to which these strategies were used 
before the advent of IPR/LA is not known. 

o Reported planning time for reading instruction had increased for 
A3 to 73 percent of the teachers, depending on the group 
surveyed, and particularly for teachers with students at different 
achievement levels. This suggests that teachers were attempting 
to adapt the program to fit the needs of students as the program 
requires. 

o Expected changes in teacher-directed, comprehension-related 
activities were not found to be widespread. The program stresses 
comprehension- related activities that require teacher direction. 
However, reading in context (reading sentences, paragraphs, and 
whole stories) did not constitute a majority of the time spent 
during observed reading instruction. Twenty-seven percent of the 
teachers interviewed felt that comprehension-related instruction 
had increased because of implementation, while fewer teachers 
(11%) reported that teacher-directed reading instruction had 
increased because of the program. 



1. Teacher-directed instruction refers to reading instruction that is 
directly under the teacher's guidance. Alternatively, students could work 
independer*-1.y, with peers, or with an aide. 
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The reading/listening program in narration stresses the use of the 
best in children's literature, with tradebooks being preferred to 
basal readers an*i workbooks. While students were observed using a 
variety of materials, the amount of time spent with basal readers 
and workbooks was more than double the time spent reading or 
listening to tradebooks. 

The use of criterion-referenced tests developed for the curriculum 
posed some significant problems. These tests were designed to 
support instruction and were strongly recommended by the 
Department of Academic Skills. However, their importance and 
uaeculness were not clear to all school staff. Forty-one percent 
of the teachers interviewed found the result* helpful. 



o Record keeping was seen au inadequate. The lack of a record- 
keeping system was seen as a serious weakness by area office 
staff. Thirty-nine percent of the teachers reported including 
progress in the program's objectives in their own records of 
Gtudents' reading achievement 

o A substantial proportion of teachers (34-39Z) did not feel the 
program was appropriate for low achievers. This feeling was more 
prevalent among teachers whose classes included such students. 
These feelings were clearly reflected in instructional practices. 
In first grade, low-achieving students received less teacher 
direction, spending more time in independent seatwork. Low- 
achieving first graders were also less likely to be exposed to 
instructional strategies recommended by the curriculum. In fourth 
grade, low-achieving students also spent less time under teacher 
direction, were more likely to use basals, and spent less time 
using tradebooks. 

SUPPORTING AND MANAGING IMPLEMENTATION 

While the Department of Academic Skills developed the new curriculum, staff 
of that department are not directly responsible for its implementation. 
They provide mainly consultative and supportive services as requested. The 
area offices are charged with monitoring the process. Ultimately, it is up 
to the individual schools to use the new program in the classroom. 

First-year data indicate a number of problems in the supports for and 
management of the program implementation process. These problems appear to 
have arisen '.ess because of the actions of any one group than because of a 
lack of clarity regarding responsibilities and expectations. No consistent 
criteria were in use systerawide to monitor program implementation. What was 
considered evidence of implementation appeared to be an individual decision 
by each supervisor and/oi teacher specialist. Neither the training nor the 
materials provided to schools to support 0.3 implementation process were 
perceived as adequate. Mora and improved in-service training was called 



2. As of May i, 1984, these tests were mandated as a part of the minority 
achievement monitoring program. 
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for. Finally, the role of the reading teacher was reported to have changed, 
and thla change created role conflicts for some school-based reading 
teachers. Specific findings were the following: 

o Ad hoc and individual criteria for monitoring implementation were 
in use; no systemwide standards were applied, and practices varied 
from area to area. Several area respondents felt implementation 
was not taken seriously because no clear statement of required 
program components was available. 

o Many area staff and principals felt that the in-service training 
provided to teachers was not sufficient. Approximately half the 
teachers had received five hours or less of training on the new 
curriculum. First grade teachers and teachers with low-achieving 
students in their class were more likely to feel the training was 
insufficient and to want additional help. 

o While almost all teachers were familiar with the instructional 
guides and found them helpful, 44 percent mentioned problems with 
their organization and presentation. Reading teachers and media 
specialists felt other materials (core-book lists and basal 
correlation sheets), while useful, were too limited in scope. 

o There is a conflict in the role of school reading teachers between 
providing direct instruction to students and serving as a resource 
to teachers and assisting principals in implementation. 

IMPLICATIONS 

The problems in implementation of the reading/language arts program have 
implications at two levels: for the curriculum itself, and for the general 
process of curriculum implementation and improvement in MCPS. 

IFR/LA Implementation 

First, with respect to IPR/LA implementation itself, a number of 
implications can be drawn from the specific study findings. These concern 
instructional practice, use of tests, in-service training and support 
materials, and program monitoring. 

Some, but not all, aspects of instructional practice matched the Board- 
mandated Program of S tudies. Continued effort is needed in order to 
increase the use of these practices. The issue is of particular concern for 
low-achieving students. Teachers have not been convinced that the program 
is appropriate for low achievers. The instruction provided to these 
students was less likely to match program standards than that provided to 
higher achievers. Since the curriculum was mandated for all students, a 
greater effort needs to be made to provide adaptations which better meet 
these students* needs, to convince teachers of the efficacy of the program 
in its present form with low achievers, or to improve management procedures 
for implementing curricula. 

The purpose of the curriculum tests and the procedures for their use are not 
well understood. These ambiguities as well as the lack of a record-keeping 
system have resulted in a mixed message about the need to assess performance 
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on curriculum objectives. These problems will be remedied in large part by 
the Inclusion of these tests in the mandated annual testing program 
initiated this spring and the provision of a record-keeping system for use 
with the program, which is expected to be available in 1984-85. Continued 
efforts need to be directed, however, toward helping schools understand how 
to use the data produced by the tests. 

Neither in-service training nor support materials have been adequate. While 
a specific number of additional hours cannot be recommended, more and/or 
improved training is clearly needed, especially on how to adapt and use the 
program with low achievers. In addition, teachers want a better 
understanding of program principles and mor* practical demonstration of 
classroom procedures using the new program. Training should include 
assistance to teachers in adapting and developing new lessons bored on the 
program objectives. Support materials need to be Improved to increase their 
usefulness: instructional guides should be Indexed by objectives, and core- 
book lists and basal correlation sheets should be updated and expanded. 

A clear statement of what should happen in the classroom is also necessary 
for effective monitoring of program implementation. Principals and teachers 
need a list of specific program components (type of plans required, 
appropriate instructional practices, use of tradebooks) to use in assessing 
the progress of implementation. However, unless this results in a statement 
of required program components, implementation may continue to be piecemeal. 

Curriculum Implementation In MCPS 

Problems such as those mentioned above are not unique to Montgomery County, 
and time alone does not provide a cure. Study af t*r study has documented 
innovative ideas that were developed but never Implemented In any but a 
superficial way. It has become clear that program implementation. is, under 
the best of circumstances, difficult and must be carefully orchestrated if 
it is to succeed. It is time to take a very serioos look at how MCPS goes 
about the program implementation task, as this is only one of a series of 
studies in which implementation problems have been found. DEA studies in 
other program areas (the Instructional System in Mathematics, career 
education, gifted and talented) have documented similar problems In the 
implementation process. In MCPS, curriculum development, monitoring, and 
instruction have been separated, and there is no system in place for 
curriculum Implementation that ties the various elements together. 
Colleglal monitoring and good will are not sufficient to accomplish 
implementation. Without systematic procedures which both support and 
enforce implementation, the result is Board-approved programs that are never 
fully implemented. 

MCPS has a number of curriculum reform efforts In various stages of 
dissemination in the schools. In the English Language Arts, the writing and 
speaking program 3 is currently being piloted, and plans call for broader 
implementation next year. Before such expansion is allowed, DEA feels it is 
critical that a complete, multlyear plan for implementation be prepared. 
Rather than continue with the "dribble" approach to program implementation 



3. Approved by the Board of Education on May 22, 1984. 
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that has been characteristic to date, it is time to take a more systematic 
approach to the problem. Without systematic procedures for implementation 
that specify responsibilities, establish requirements, and provide 
resources, there is no reason to expect that the writing program will fare 
any better than those examined to date. 
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READING STUDY: FIRST-YEAR REPORT 



CHAPTER I: INTRODUCTION 



In 1976, MCPS began the development of the K-8 Instructional Program in 
Reading/Language Arts (IPR/LA) for teaching the English Language Arts. 
Goals of the effort were to improve reading/listening comprehension 
Instruction and to Improve program consistency across grades, schools, and 
areas. The objectives for the new program were approved by the Board of 
Education In June of 1981 as the mandated Program of Studies for 
reading/listening for all students In Grades K-8. The IPL/LA Includes the 
mandated objectives and a variety of recommended support materials (e.g., 
teachers 9 guides and tests). Materials for the narration component have 
been completed, while those for other forms of discourse (exposition, 
persuasion) are In various stages of Implementation. The program Is 
currently being Implemented In schools under the direction of area offices. 
The Reading Study Is a multlyear evaluation of the narration component of 
the IPR/LA. Its purpose Is to systematically judge the degree to which 
schools are actually using the new curriculum and, once In use, Its Impact 
on reading Instruction and student achievement. Based on study data, 
recommendations can be made to pft'ect necessary adjustments to the 
reading/language arts program and to Improve the administrative structure 
for Implementation of Instructional programs In other areas. 

The first full year of data collection for the Reading Study was 1982-83. 
This progress report on the first year of the study focuses on descriptive 
Information about the characteristics of reading Instruction and problems In 
the Implementation process In the study schools. As more data are 
collected, an attempt will be made to develop school level profiles that 
build on this Information and clarify how the various characteristics of 
Implementation relate to each other. This first chapter briefly summarizes 
the study design. Chapters II and III present the findings on Instructional 
practices In reading, and the program Implementation process and problems. 

WHAT WAS STUDIED 

The first-year report focuses on several policy Issues that can be addressed 
using the data already collected. The three major Issues examined In this 
report are the following: 

o Is IPR/LA being Implemented for all students? 

o Are the supports, training, and materials necessary for 
Implementation being provided to school personnel? 

o What roles are played by the different administrative levels In 
monitoring and managing the Implementation process? 

The first Issue concerns how Implementation of the new program has affected 
what actually happens In the classroom. In collecting data, the following 
research questions were addressed: * 
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o What were the characteristics of the reading instruction provided 
to students and how did the.se match up with the new curriculum as 
described in the Program of Studies ? 

o Have teachers developed a coherent approach to reading instruction 
that encompasses those instructional components emphasized in the 
new program? 

o Has this new approach been implemented with all students, 
regardless of achievement level? To what extent were variations 
in the degree of implementation related to the achievement 
composition of schools or classes? 

First-year data also permit an analysis of the supports for the 
implementation process. The research questions Included the following: 

o What training and materials have been provided to schools to 
implement the program? 

o Has the in-service training provided been adequate to support 
program implementation? 

o Were the support materials appropriate, and what problems have 
occurred in their use? 

First-year data also provide a description of the administrative process. 
MCPS is a large and complex administrative system. The Department of 
Academic Skills developed the new curriculum, but they are not responsible 
for its implementation. Area offices are supposed to support and monitor 
the process, while It is ultimately up to the Individual schools to use the 
new program In the classroom. Research questions focused on the management 
and administrative problems that may arise out of the effort to reorient 
classroom practice and Included the following: 

o Were procedures In place to monitor the Implementation of the new 
curriculum? Were standard criteria for assessing Implementation 
applied? 

o What roles were played by different school staff In monitoring and 
managing the implementation process? 

A sauple of 20 MCPS elementary schools was selected for the reading study 
and stratified on the basis of achievement level and length of tlire the 
Tchool has been exposed to IPR/LA. Schools from each area were included. 
The stratlfiers were selected as proxies for variables that may Influence 
implementation and Its impact and about which data are being collected 
during the study. The number of years Implementing serves as an Initial 
proxy for degree of Implementation. Two schools In the sample had not 
officially begun Implementation and were originally Included to provide some 
baseline information on classroom characteristics prior to program 
Implementation. (Data collection revealed, however, that many teachers In 
these schools had already used the program prior to the study. Thus, their 
classroom practices could not be considered descriptive of those prior to 
Implementation.) Since It was thought that program implementation and the 
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effect on Instruction might vary by achievement level, the sample was also 
stratified by achievement. Schools were classified as above or below the 
county mean on the previous year's CAT reading achievement scores. Within 
the sample schools (see Table 1 for a list of study schools by strata), four 
classes (2 first grade and 2 fourth grade) were randomly selected for study 
the first year. The approximately 1800 sample students In thes? classes 
will be followed for the duration of the study. Classrooms at these two 
grade levels were selected In order to allow comparisons of program 
Implementation and Instructional changes at the primary and upper elementary 
grades. 

Various data collection Instruments were developed and used during the first 
year of the study, Including Interview guides, questionnaires, and 
observation checklists. Topics covered In the Interviews with first- and 
fourth grade teachers Included planning, in-service training, the role of 
school-based specialists In Implementation, the use of curriculum support 
materials, Instructional practices In reading and changes due to the 
curriculum, attitudes toward the program, and Its appropriateness for 
students at all achievement levels. Questionnaires given to classroom 
teachers at other grade levels In the sample schools covered the same topics 
In less detail. In order to get a clearer picture of Instructional 
practices, unannounced classroom observations were conducted In the sample 
classrooms during three time periods (fall, winter, and spring). Both 
teacher and student behaviors were observed. The Information has been 
reported In group terms to preserve teacher confidentiality. No observation 
data have been made available In any way for teacher evaluation purposes. 
The technical appendix describes the data collection procedures and the 
categories used for classroom observations In more detail. 



CHAPTER II: PROGRAM IMPLEMENTATION 

The first major policy Issue addressed In this report Is whether the new 
reading/language arts curriculum Is being Implemented for all students. 
First-year study data present a mixed picture. Some aspects of reading 
instruction more closely resembled the Hoard-approved Program of Studies 
than others. For some teachers, the program did appear to constitute an 
alternative approach to teaching reading that encompasses different 
Instructional strategies as well as the way basal readers are used. However, 
the program was not being Implemented for students at all achievement 
levels; low-achieving students were receiving different Instruction. These 
variations were not related to the amount of time a school had officially 
been Implementing the program. The findings are presented In the following 
sections. 

INSTRUCTIONAL PRACTICES 

Information was collected on a variety of components of reading instruction, 
with particular attention being given to those believed to be affected the 
most significantly by the reading/listening program in narration. The 
degree of program Implementation appeared to be uneven: 
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TABLE 1 



Sample Schools for the Reading Study 



Years Implementing 
Curriculum 


Achievement Level 
Below County i Above County 
Mean I Mean 


Total 


Since 1981-82 


Broad Acres 
Maryvale 
Rolling Terrace 


Potomac 

Fallsmead 

Sherwood 


6 


From 1980-81 


*Beall 
Fields Road 
Vlers Mill 


Seven Locks 
Cedar Grove 
Greenwood 


6 


Prior to 1980 


Poolesvllle 

Clarksburg 

Glenallan 


Bannockburn 
Laytonsvllle 
Kemp Mill 


6 


Nonlmplementlng Schools 




Bethesda 
Farmland 


2 
20 


*Beall consolidated with a school In this category of Implementation. 
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o Reported planning time for reading Instruction had Increased for 
43 to 73 percent of the teachers, depending on the group 
surveyed, and particularly for teachers with students at different 
achievement levels. This suggests that teachers were attempting 
to adapt the program to fit tne needs of students as the program 
requires. 

o Expected changes In teacher-directed, comprehension-related 
activities were not found to be widespread. The program stresses 
comprehension- related activities that require teacher direction. 
However, reading In context (reading sentences, paragraphs, or 
whole stories) did not constitute a majority of the time during 
observed reading instruction. Twenty-seven percent of the 
teachers interviewed felt that comprehension-related instruction 
had increased because of implementation, while fewer teachers 
(HZ) reported that teacher-directed reading instruction had 
increased because of the program. 

o The reiding/liitenlng program in narration stresses the use of the 
best In children's literature, with tradebooks being preferred to 
basal readers and workbooks. While students were observed using a 
variety of materials, the amount of time spent with basal readers 
and wotkbooks was more than double the time spent reading or 
listening to tradebooks. 

o Instructional strategies recommended by the curriculum guides 
(setting a purpose for reading, using students' background 
knowledge, and open-ended questioning) were In fairly widespread 
use. Teachers were observed using these strategies with S3 to 77 
percent of the groups in reading instruction. The degree to which 
these strategies were used before the advent of IPR/LA Is not 
known. 

o The use of criterion-referenced tests developed for the curriculum 
posed some significant problems. These tests were designed to 
support Instruction and were strongly recommended by the 
Department of Academic Skills. 2 However, their importance and 
usefulness were not clear to all school staff. Forty-one percent 
of the teachers Interviewed found the results helpful. 

o Record keeping was seen as Inadequate. The lack of a record- 
keeping system was seen as a serious weakness by area office 
staff. Thirty-nine percent of the teachers reported including 
progress lu the program's objectives in their own records of 
students' reading achievement. 



1. Teacher-directed instruction refers to reading instruction that Is 
directly under the teacher's guidance. Alternatively, students could work 
Independently, with peers, or with an aide. 

2. As of May 1, 1984, these tests were mandated as a part of the systemwide 
testing program for monitoring minority achievement. 
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o Teachers who reported using the approach to Instructional 
materials preferred by the curriculum designers (16Z to 32Z) were 
observed providing reading i nut ruction that matched program design 
more closely In other respects as well: teacher direction, time on 
comprehension, and Instructional strategies. 

Details In each of these areas are presented In the following sections. 
Instructional Planning 

The Program of Studies requires that teachers select a variety of materials 
and adapt strategies and suggested lessons to fit the needs of their 
students. In other words, one cannot just open the teacher's manual and 
Implement the program without additional Independent planning. Thus, one 
would expect the teacher planning time to Increase with Implementation. 
Teachers were asked about their Instructional planning for reading and 
whether Implementation of the curriculum has had an Impact on that activity. 
Forty-three percent of the first and fourth grade teachers Interviewed In 
the fall felt that the amount of time they spent on Instructional planning 
for reading had Increased as a result of program Implementation. The 
response to the teacher questionnaire Indicated an even higher percentage 
(73) felt that planning time had Increased due to the new curriculum . 
Teachers with students at different achievement levels were more likely to 
feel the time spent on planning had Increased as a result of the curriculum 
than were those teachers whose students were all at the same achievement 
level (57Z vs 42Z). 

Teacher-Directed Instruction and Comprehension 

Table 2 presents some data on reading Instructional characteristics observed 
In the first and fourth grade classrooms: teacher direction, reading tasks, 
materials used, and Instructional strategies. Because of the curriculum's 
focus on comprehension activities that Involve critical thinking and 
discussion of what has been read, larger amounts of time may be spent In 
teacher-directed Instruction when the program Is Implemented. As Table 2 
Indicates, students In Grades 1 aud 4 were working under teacher direction 
approximately 47 percent of the time during the observed reading 
Instruction. However, In the Interviews, only 11 percent of the teachers 
felt that this amount of time had changed as a result or the program. 

The curriculum Is Intended co Increase the time spent on such comprehension 
activities as reading in context (reading sentences, paragraphs^ or whole 
stories) and to reduce the time students spend on decoding isolated letters 
and words. Table 2 indicates that over the school year, as might be 
expected, comprehension activities increased, while the proportion of time 
spent on letters and words declined. However, reading in context took up 
less than half the time in reading instruction in either grade. In the 
interviews, 27 percent of the teachers felt they spent more time on 
comprehension skills because of the new program. 
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TABLE 2 

Characteristics of Reading Instruction 
(X of student's observed time In reading Instruction) 







First Grade 

* XML. QUC 




Working Under Teacher Direction 




47 


47 


Working Independently 




50 


42 


Working with Other Adults 




2 


8 


Working with Peers 




1 


3 


Reading Tasks: 








Letters and Words 


Nov. 


33 


10 




Feb. 


30 


9 




May 


18 


9 


Reading In Context 

w-^^-^v^v ^m mm^^ mm mm mm V mwt W 


Nov. 


16 




(sentences, paragraphs, 
or stories) 








Feb. 


18 


28 




May 


34 


42 


Listening* 




26 


38 


Writing** 




22 


11 


Speaking 




5 


6 



♦Listening activities Included listening to the teacher or peers reading 
aloud as well as listening to teacher directions. 

**The definition of writing activities was a broad one and Included copying 
letters and words, note-taking, etc., as well as creative writing. 
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TABLE 2 (continued) 

Characteristics of Reading Instruction 
(Z of student's observed time In reading Instruction) 





First Grade 


Fourth Grade 


Materials Used: 






Basal Readers 


20 


22 


Workbooks /Worksheet s 


31 


21 


Other Materials 

(boardvork, film, teacher-made 

materials ) 


AO 


25 


Trade book s 


5 


26 


No Materials 


4 


6 


Instructional Strategies: 


(Z of groups observed)* 


Setting Purpose 


66 


70 


Use of Student Background and 
Experience 


38 


67 


Open-ended Questions 


66 


83 



*Thls figure represents the proportion of all reading groupt observed 
across the three time periods with whom teachers used a strategy. Since 
teachers could use more than one strategy, the percentages total more than 
100 percent. 
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Variety of Materials 

One of the major aspects of the new reading/language a~ts program In 
narration Is Its focus on the use of a variety of materials and the best In 
children's literature. Observation data were therefore gathered about the 
kinds of materials In use during reading Instruction. As Table 2 Indicates, 
the proportion of time In reading Instruction that students spent using the 
basal readers was approximately 20 percent for both first and fourth 
graders. Teachers were observed using them with a majority of their reading 
groups In both grades. It Is Important to note that the use of workbooks Is 
not encouraged In the curriculum; yet It did occupy a substantial proportion 
of Instructional time. Conversely, the proportion of time spent using 
tradebooka was generally low at both grade levels. Their use was almost 
nonexistent at the first grade (5%), while In the fourth grade, students 
were observed using t i ad e books 26 percent of the time. 

Instructional Strategies 

Certain Instructional strategies (setting a purpose, using student's back- 
ground, open-ended questions) are used In the sample lesson units In the 
Instructional guides and are ones that the program developers would like to 
encourage. The use of these practices In Montgomery County prior to the 
advent of IPR/LA Is not known, and thus their use by teachers does not In 
Itself Imply use of the guides or changes relating to the new curriculum. 
Such practices were generally rather widespread In study schools, as Table 2 
Indicates. Teachers were observed setting a purpose for group Instruction - 
"Now let's read to find out what a character In the story does next 1 ' - with 
72 percent of the groups observed. The teacher's use of students' 
background knowledge and experience In the lesson was more likely to take 
place In fourth grade than In first grade. Overall, teachers were observed 
using this strategy with just about half (53%) of the groups observed during 
reading Instruction. Teachers used open-ended questions to encourage 
critical thinking by their students with an average of 77 percent of the 
groups observed. First grade teachers used such questions somewhat less 
frequently than fourth grade teachers. 

Testing and Record Keeping 

A final aspect of Instructional practice about which data were collected 
concerns the evaluation of student progress In learning the re id Ing/ language 
art' objectives. Criterion-referenced tests at each grade level have been 
developed for the program. These tests provide a means for teachers to 
assess students' performance on the curriculum objectives In comprehension 
(decoding objectives are not covered by the tests because of the wide 
availability of assessment measures for such objectives) and were to be 
given In the fall and winter at the beginning of each semester. The tests 
are not required as part of the curriculum but are certainly strongly 
recommended by DAS, and It Is hard to see how a program could be fully 
Implemented without having some measure of student performance on its 
objectives. Nevertheless, the tests have been a source of controversy and 
problems Table 3 provides some data on their u~e and reactions to the 
tests by various respondents. Forty-one to 63 percent of classroom 
teachers found the test results helpful. A number of substantive concerns 
were expressed by teachers and other staff during the Interviews. 



21 



9 



* ) 



First, the lack of any phonics/decoding on the test has confused teachers 
about the place of such skills in the curriculum. Teachers Interpreted the 
absence as a lack of emphasis on these skills, although they are included in 
the Program of Studies . This is a clear example of the problems of 
communication between program developers and those who are supposed to 
implement the program. Second, some concerns were expressed about the 
appropriateness of the CRTs for the first grade and for poor readers in 
general. The proportion of first graders exempted from the testing by 
teachers varied widely in the sample schools, and in one school only two 
students were tested. 

Confusion also existed about ou t-of-grade-level testing. The tests were 
designed to measure performance on curriculum objectives at the student's 
instructional level. If all students are tested at grade level, the tests 
may be too easy for many above- grade- level students and too difficult for 
those reading below, and they were not Intended for use in this way. While 
out-of-level testing thus can be done and makes more sense with this 
curriculum since the broader objectives span grades, such tests could not 
be centrally scored which gave many schools and teachers the Impression it 
should not be done at all. Teachers with below-grade-le/el students were 
forced to hand score out-of-level tests or were left without any information 
on those students' performance on the curriculum objectives to use for 
instructional planning. 

It is clear that a better understanding of the tests and their purposes as 
well as procedures for their use are needed at the school level. It is 
interesting to note the differences between reading teachers' and classroom 
teachers' views of the tests (see Table 3). Presumably the reading teachers 
have a clearer view of their uses, but unfortunately this has not been 
communicated to classroom staff. Only 6 percent of the teachers interviewed 
recalled receiving any in-service training on the use of the tests or 
interpretation of test results. 

The mixed message about the need to assess performance on curriculum 
objectives was compounded by the lack of a record-keeping system developed 
for the program. Twelve of the 15 area respondents felt that this lack 
created major problems for the schools in implementing the program. Without 
a countywide system, it is difficult to transfer information on student 
performance from school to school, a particular problem when schools 
consolidate or with high mobility students. Without a record-keeping system 
that reflects the objectives and books read, teachers have less incentive to 
consider the new program important. They continue to use the basal and to 
record student progress in terms of the basal reader system provided. Only 
39 percent of those responding to the teacher questionnaire included 
curriculum objectives from the Program of Studies in their records. A few 
area respondents questioned how the program could be "ready to go" when such 
mechanisms were not in place. 



3. A record-keeping system was piloted for several years and was approved 
for general use in 1984-85. It is recommended by DAS for use with the 
curriculum. 
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TABLE 3 

Reported Use of Curriculum Tests by School Staff 



Percentage of respondents answering yes: 
l&A Grade Teachers In Principals Reading 
Teachers Other Grades Teachers 



CRTs: 

Use 68 
Timely* 

Helpful 41 

Novel tests: 
(4-6 Grade Only) 

Use 58 

Helpful 44 



86 89 100 

77 61 

63 56 86 

50 95 

46 83 



Blanks indicate that the question/ response categories were not Included in 
the instruments for those respondents. 

♦Timeliness referred to the receipt of test materials. 



23 

11 



A Coherent Approach 

The curriculum guides suggest three different ways in which teachers may use 
materialr + j approach teaching the program's objectives: 1) the teacher 
uses a basal reader in sequence and supplements it where needed with other 
materials in order to teach IPR/LA objectives, 2) the teacher does not use a 
basal in sequence but primarily selects appropriate basal stories and 
supplements with other materials to teach IPR/LA objectives, and 3) the 
teacher selects IPR/LA objectives and finds the best materials from various 
sources to teach the objectives and fit student needs. While all three are 
in the guide, the third alternative fits most closely the ideal envisioned 
by program developers. These choices stress the way in which basal readers 
are used, aH.no ugh other program aspects also distinguish the new 
curriculum from previous approaches. The study findings suggest that the 
approach to instructional materials is Integrated with other components of 
instruction and that these elements form a coherent program for reading 
instruction. 

The proportions of different respondents selecting each of the three 
approaches to teaching the curriculum objectives are presented in Table 4. 
Six percent of the first grade teachers and 26 percent of the fourth grade 
teachers in the interviews, and 32 percent of the classroom teachers in 
other grades who responded to the questionnaires, selected the ap n ach 
preferred by the program designers. The choice of approach was c> .cly 
associated with various instructional practices observed in use by these 
teachers: teacher direction, reading whole stories, and instructional 
strategies. Tables 5 through 7 present information on the relationship 
between the approach a teacher selected in the interview and these other 
aspects. of observed instruction. As Tables 5 and 6 indicate, teachers 
selecting the third approach— the one preferred by program developers — 
provided more teacher-directed instruction, and their students spent more 
time reading whole stories. The instructional strategies used also varied 
depending on the choice of approach to instructional materials. Three 
strategies associated with the curriculum — setting purposes, using students' 
background knowledge and experience, and asking open-ended questions— were 
indeed more prevalent in the classrooms of those teachers who selected the 
curriculum approach (see Table 7). Thus, greater teacher direction, more 
time spent reading whole stories, and the use of instructional strategies 
that emphasize meaning in discourse and critical thinking did seem to 
characterize classrooms in which the curriculum was being implemented. 



4. These options are described on pj of the General Introduction to the 
guides under the heading Use of Materials. 

5. Classroom teachers were asked to select the approach that they used; 
other respondents were asked to select the approach they felt was most 
appropriate for teachers. 
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TABLE 4 

Approaches to Instructional Materials 



Percentage of Respondents Selecting Each: 





Basal In 
Sequence 


Basal 

Selectively 


Objectives and 
Best Materials 


Other 


Area Supervisors 


0 


12 


88 




Area Teacher Specialists 


33 


33 


33 




First Grade Teachers 


83 


11 


6 




Fourth Grade Teachers 


43 


26 


26 




Teachers In Other Grades 


45 


15 


32 


8 


Reading Teachers 


19 


19 


48 


14 



TABLE 5 



Observed Tine Spent In Teacher-Directed Reading Instruction by Students 
of Teachers Selecting Different Approaches to Instructional Materials 



Amount of Observed Time 
In Teacher-Directed 
Instruction: 



Low 



Medium 



High 



(Percentage of Students) 



Teacher Selected: 

Basal In Sequence 
Basal Selectively 
Objectives and Best Materials 



42 
28 
17 



31 
44 
25 



28 
28 
58 



13 

OK 



1 



TABLE 6 

Observed Time Spent Reading Whole Stories by Students of Teachers Selecting 
Different Approaches to Instructional Materials 



Amount of Observed Time 
In Reading Whole Stories: 



Low 



Medium 



High 



Teacher Selected: 

Basal In Sequence 
Basal Selectively 
Objectives and Best Materials 



(Percentage of Students) 



43 
25 



42 
34 
33 



15 
25 
42 



TABLE 7 

Instructional Strategies Observed in Use by Teachers Selecting Different 

Approaches to Instructional Materials 

Strategies 



Set Purpose Use Background Open-ended 

Knowledge Questions 





(Percentage of Teachers Observed Using With:) 






No Some All 
Reading Groups 


No 


Some 


All 


No Some 


All 


Teacher Selected: 


















Basal In Sequence 


12 


48 


40 


36 


40 


24 


12 44 


44 


Basal Selectively 


25 


0 


75 


75 


25 


0 


0 25 


75 


Objectives and 
Best Materials 




0 


84 


17 


17 


66 


16 0 


84 
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INSTRUCTIONAL DIFFERENCES FOR LOW ACHIEVERS 



A final Issue concerning Implementation of the curriculum at the classroom 
level Is whether the program Is perceived as appropriate and Is being 
Implemented for students at all achievement levels. The first-year data 
Indicate that different Instruction, perhaps even different reading 
programs, were being provided to students at different achievement levels: 

o A substantial proportion of teachers (34 to 39 percent) did not 
feel the program was appropriate for low a hievers. This feeling 
was more prevalent among teachers whose classes Included such 
students. 

o In first grade, low-achieving students received less teacher 
direction, spending more time in Independent seatwork. 

o Low-achieving tirst graders were also less likely to be exposed to 
Instructional strategies recommended by the curriculum. 

o In the fourth grade, low-achieving students also spent less time 
under teacher direction, were more likely to use basals, and spent 
less time using tradebooks. 

While some of the differences In first grade may be attributed to the fact 
that below-grade students were frequently nonreaders with the obvious 
attendant differences In appropriate Instruction, other differences such as 
amount of teacher direction seem less appropriate. The fourth grade 
differences do appear to Indicate that at least some teachers were not 
attempting to Implement all aspects of the Program of Studies with their 
low-achieving students. 

Table 8 presents respondents views on the appropriateness of the curriculum 
for students of differing achievement levels. Area staff were mcjt likely 
to feel the program could work for all students, Including those below grade 
level. However, only 48 to 56 percent of classroom teachers felt the 
program was appropriate for the low achievers. In commenting on the 
problems In Implementing the program with students who are achieving below 
grade level, area staff raised a number of concerns that were also reflected 
in teacher Interviews. "Teachers don't feel comfortable using the program 
with this group and are reluctant to attempt It. They feel below-grade 
students must concentrate on basic skills, I.e., decoding and phonics, an<i 
can 9 t cope with comprehension." One area staffer noted that teachers do not 
expect lower achievers to be able to handle comprehension skills, so they 
don 9 t teach them. This creates a self-fulfilling prophecy when the students 
do badly on comprehension tests. Area staff also suggested that more sample 
units should be Included In the guides to show teachers how to make 
adjustments for the lower-achieving groups. Such units are Included for the 
gifted and talented at the other end of the scale. The most frequent 
comment from the teacher Interviews was that these students must stick to 
the basics, a statement that suggests they viewed the program as at most a 
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TABLE 8 

Appropriateness of Program for All Students 



Percentage answering "Yes, program meets needs of 
students who are": 

Above Grade On Grade Below Grade 



Area Staff 


100 


100 


87 


Principals 


89 


89 


61 


144 Grade Teachers 


96 


87 


56 


Other Teachers 


88 


89 


48 


Reading Teachers 


100 


100 


38 


Media Specialists 


67 


67 


44 



TABLE 9 

Relationship Between Perceived Appropriateness of Program for All 
St«"!ent8 and Class Achievement Composition 

Percentage of Teachers 

In your opinion $ does IPR/LA meet the needs of 
below-grade-level students? 

Yes No Don 9 t Know 

Class Composition: 



Students all above grade 


50 


0 


50 


Students all on grade 


65 


23 


12 


Students above or on grade 


43 


43 


14 


Students Include those below grade 


43 


46 


11 
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supplement for the be low-grade-level students. 

Teacher's views on whether the curriculum meets the needs of low-achieving 
students seemed to be associated with the composition of their classrooms. 
The data presented In Table 9 suggest that teachers with lower-achieving 
students In their class war'; more likely to have concerns about the 
program's appropriateness for the below-grade- level students. Neither the 
number of years a school had officially been imple" ~Mng the program nor 
the amount (number of hours) of training a teacher had received was 
associated with teachers' views of the program's appropriateness for low 
achievers . 

Observation data broken down by the achievement level of the student reading 
groups are presented In Tables 10 to 13. In classifying reading groups' 
achievement levels, the level of the book the group was reading In February 
was used. Students In pullout ESOL or other resource/special education 
programs who did not receive regular classroom reading Instruction were not 
Included In the observations. As the data In Tables 10 to 13 Indicate, In a 
number of Instructional areas Important for Implementation of the new 
curriculum, the Instruction provided to lower-achieving students was less 
likely to match up with program characteristics. The amount of time under 
teacher direction (Tables 12 and 13) Is one aspect of instruction where the 
data reveal considerable variation depending on the reading level of the 
group to which the student belongs. While the curriculum would encourage 
more teacher direction generally, at both grade levels a much higher 
proportion of time was teacher directed for the higher-achieving students. 
The materials used for reading Instruction were also observed to vary 
depending on the achievement level of the reading group, as well as by 
grade. Basal readers were generally in use with more groups at first grade 
which is not surprising. In this grade, however, low achievers were more 
likely to be found using other teacher-made materials. In the fourth grade, 
the low achievers used the workbooks and worksheets associated with the 
basals mc*e of the time and spent 10 percent more time than higher groups 
using basal readers. 

Conversely, tradebooks were used less with the low-achieving students. They 
were scarcely used at all in the first grade for any groups; however, in the 
fourth grade the proportion of groups at different achievement levels 
observed using the tradebooks varied greatly. Only 17 percent of the low- 
achieving groups were observed with such materials, while 24 percent of the 
on-grade level groups were observed using tradebooks; and 41 percent of the 
above-grade-level reading groups used tradebooks (Table 11). Expressed as a 
proportion of time, the higher achieving groups spent approximately 8 
percent more time using such books than the below-grade-level groups (Table 
13). Board-mandated exposure to a variety of materials Is thus less likely 
for the lower-achieving students, perhaps reflecting in part the attitudes 
reported above about the appropriateness of the curriculum for such 
students. 



6. Low expectations creating self-fulfilling prophecies, low achievers not 
being taught certain skills, etc., are phenomena found in many studies of 
low achievers. See, for example, G.E. Mendels and J. P. Flanders, Teacher 
Expectations and Pupil Performance. American Research Journal. 1973. 10. 
203-212 ' — ' 
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TABLE 10 

Grade 1: Materials and Instructional Strategies Used With 
Students at Different Achievement Levels* 




Basal 
Readers 



Other 

Mats. 
*** 



MATERIALS 
0 * Be t ow grade 



radebi<s 




: ab I i eh Use Stu 
'urpose Bkgrd 
& Exp, 



Use Open 
ended 
Ques . 



INSTRUCTIONAL STRATEGIES 



□ 



■ On grade 



= Above Qrade 
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*In classifying reading groups as above, on, or below grade, Information 
provided by teachers about the reading level of the groups In February 
was used. Teachers Indicated the level of the books the group was 
reading In and whether they considered the group to be above, on, or 
below grade. 

** This figure represents the proportion of all groups at that level that 
were observed whose Instruction had a particular characteristic. Since 
the characteristics were not mutually exclusive, the percentages total 
more than 100 percent. 

***0ther materials Included workbooks, dittos, and tea char- ma d' materials. 
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TABLE 11 
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Grade 4: Materials and Inatructional Strategies Used With 
Students st Different Achievement Levels* 
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MATERIALS 



Below grade 



□ 
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INSTRUCTIONAL STRATEGIES 
" Above grade 
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*In classifying reading groups as above, on, or below grada, inforraatio i 
provided by teachers about the reading level of the groups i February 
was used. Teachers indicated the level of the books the group was 
reading in and whether they considered the group to be above, on, or 
below grade. 

** This figure represents the proportion of sll groups at that level that 
were observed whose instruction had a particular characteristic. Since 
the characteristics were not mutually exclusive, the percentages total 
more thau 100 percent. 

***0ther materials included workbooks, dittos, and teacher-made materials. 
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TABLE 12 
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Grade 1: Characteristics of Instructional Time Spent 
by Students at Different Achievement Levels* 



cher 
Dir. 



ndep CS 



ters Rdg i n 
& Context 
Words- 



i st . 



TASKS 

0 = Below grade CD 



*asal s 
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*In classifying reading groups as above, on, or below grade level, 
information provided by teachers about the reading level of the groups in 
February was used. Teschers indicated the level of the book the group 
was reading in and whether they, considered the group to above, on, or 
below grade level. 

+*Other materials included board work and teacher-made materials. 
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TABLE 13 
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Grade 4: Characteristics of Instructional Time Spent 
by Students at Different Achievement Levels* 
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Dir. 
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0 
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*In classifying reading groups as above, on, or below grade level, 
information provided by teachers about the reading level of the groups in 
February was used. Teachers indicated the level of the book the group 
was reading in and whether they, considered the group to above, on, or 
below grade level. 

**Other materials included board work and teacher-made materials. 
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There were also differences in the instructional strategies used with groups 
at different reading levels In first grade. First grade teachers set a 
purpose for group Instruction with more above- and on- grade- level groups. 
Teachers also used students 9 background knowledge and experience less with 
lower-achieving students. First grade teachers were likely to ask open-ended 
questions that encourage critical thinking more frequently with the higher- 
than the lower-achieving reading groups; but at fourth grade, no consistent 
differences were observed between the use of these strategies with students 
at different reading achievement levels. The extent to which these 
strategies would have differed prior to the introduction of IPR/LA is not 
known. 

■ 

The data on the differences in the instruction provided to students at 
different achievement levels should be of considerable concern since the 
curriculum was Intended to be Implemented with all students in the county. 
Overall, approximately 40 percent of the K-8 teachers who responded to 
Interviews and questionnaires had at least some students they considered to 
be reading below grade level in their class at the beginning of the year. 

The data make clear that some teachers were not fully implementing the 
program with these students; whether they vera not doing so because they 
tried and it didn't work or because of the preexisting attitudes is 
impossible to tell from study data. Study data from subsequent years may be 
used to address the question of how performance of low achievers is affected 
when the curriculum is implemented. Since only 38 percent of the reading 
teachers who do the in-service training in the schools felt the program met 
the needs of low achievers, this is a problem of implementation that will 
not be easily solved. Possible options include more training to convince 
teachers of the efficacy of the program in its present form with low 
achievers or the development of adaptations which better meet these 
students 9 needs. In addition, this problem raises the management issue of 
who should be held responsible for ensuring that the Board-mandated program 
is implemented for all students. The next chapter looks at the role of 
various levels in the implementation process. 



CHAPTER III: SUPPORTING AND MANAGING IMPLEMENTATION 

Research on implementation in a variety of areas and types of school systems 
have all documented the crucial importance of certain management tasks in 
the s,ccess or failure of change efforts. These include at a minimum 
clearly defining expectations of what is to be done, providing support for 
the change, and specifying the roles various individuals must play in the 
implementation process/ The Reading Study has collected information on 
several of these management issues. 

While the Department of Academic Skills enveloped the new curriculum, staff 
of that department are not directly responsible for its implementation. 



7. For a recent review of the implementation literature, see Susan Loucks- 
Horsley and Pat L. Cox "It's All in the Doing: Uhat Recent Research Says 
About Implementation, 11 AERA, New Orleans, April, 1984. 
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They provide training to area office staff and school-based specialists and 
consultative services as requested. They also produce a variety of 
materials to support the reading/ language arts program. The area offices 
are charged with supporting and monitoring the implementation process. 
Ultimately, it is up to the individual schools to use the new program in the 
classroom. First-year study data indicate a number of problems in the 
supports for and management of the program implementation process. These 
problems appear to have arisen less because of the actions of any one group 
than because of a lack of clarity regarding responsibilities and 
expectations. Neither the training nor the materials provided to schools to 
support the implementation process were perceived as adequate. No 
cons is tent cri teria were in use bj s tern wide to moni tor program 
implementation. Implementation tasks have created role conflicts for some 
school-based reading teachers. Details on these findings are presented in 
the following sections. 

SUPPORTS FOR IMPLEMENTATION 

In looking at the availability of the necessary support* for implementation, 
the concern was the degree to which in-service training and materials were 
provided to the schools in a sufficient and timely manner. The study found 
that while both training and materials have been provided to support schools 
and teachers, the adequacy of these supports was clearly questioned. With 
regard to training, the study found: 

o Many staff and principals felt that the in-service training 
provided to teachers was not sufficient. 

o Approximately half the teachers had received five hours or less of 
training on the new curriculum. 

o First grade teachers and teachers with low-achieving students in 
their class were more likely to feel the training was insufficient 
and want additional help. 

Data collected on the use of the curriculum materials developed for the 
program such as the instructional guides, core book lists, and basal 
correlation sheets suggest a mixed picture. The study found that the 
curriculum support materials were available and being used in the schools; 
however, problems persist: 

o While almost all teachers were familiar with the instructional 
guides and found them helpful, 44 percent mentioned problems with 
their organization and presentation. 

o Reading teachers and media specialists felt other materials (core 
book lists and basal correlation sheets), while useful, were too 
limited in scope. 

In-service Training 

All respondents were asked a series of questions about the in-service 
training which has accompanied implementation of the new curriculum. The 
need for training in the new program existed at all levels in the system. 
Difficulties arose from efforts to "provide training at the same time that 



ERiC 



23 35 



1 I 



Implementation was supposedly taking place and from the assumption that each 
level would train the one below. In reflecting on the training given to the 
schools, area staff generally felt that the training they were providing was 
not sufficient (80Z). Supervisors and specialists were assigned to serve so 
many schools that they did not feel they had adequate time to provide 
sufficient training for effective implementation. A few area 
supervisors/specialists felt that principals had not received sufficient 
training on their role in program implementation, although all principals 
reported receiving some training on the new curriculum. Area office staff 
also suggested that teachers need released time to receive in-service 
training, to visit other classrooms in schools that are successfully 
implementing the program, to see demonstration lessons, and to interact and 
exchange ideas with other teachers. 

As Table 14 indicates, most teachers felt that the in-service training 
provided had been useful. In the interviews, 56 percent of the first and 
fourth grade teachers reported the training they received was not 
sufficient. Questionnaire data from other classroom teachers indicate 25 
percent of these teachers also felt the training was insufficient to 
successfully implement the program. While the data, therefore, indicate 
differences among respondents about the sufficiency of the training £er se, 
many respondents commented on the need for other emphases in the training. 
The training received most frequently consisted of reviewing objectives ai d 
materials, assembling the Instructional guides or placing index tabs in the 
guides, and being given an overview of instructional approaches. Many 
teachers wanted additional help in the form of demonstrations and workshops, 
as Table 14 Indicates. In addition, both area staff and teachers (in 
interview comments) frequently noted the superficial and procedural nature 
of much of this training and the need to provide a better understanding of 
the conceptual basis for the curriculum and the purpose of changes in 
instructional practices. Comments written on school reading teacher 
questionnaires suggest that in a few cases, reading teachers felt 
insufficiently prepared to offer much substantive training to teachers, 
hence the focus on the organization of the guide and reviewing the 
objectives. In other cases, reading teachers indicated they reviewed just 
about every aspect of the curriculum in the training they provided, but 
given the available time, this could hardly be anything but superficial. 

Forty-nine percent of the teachers had received five hours or less of 
training on the new curriculum, 34 percent had 6 to 10 hours, and 15 percent 
had received more than 10 hours of in-service training on the program. As 
might be expected, the number of hours of training was positively associated 
with teachers' perceptions of the need for additional help or whether or not 
they considered the training >ufficlent (see Table 15). Two other factors 
besides amount of training also seemed to be related to teachers' feelings 
about the in-service training they had received. These factors were grade 
level and class composition. The first grade teachers Interviewed were less 
likely than the fourth grade teachers to feel that they had received 
sufficient training (36Z versus 51Z). The first grade teachers did receive 
less training; 58 percent of those interviewed had received less than 5 
hours of training, while only 37 percent of the fourth grade teachers had 
five hours or less. First grade teachers may also have had greater 
difficulties in using the curriculum (i.e., teachers have traditionally 
relied more on the basal and phonics at that level, so the curriculum 
represents more of a change In focus). When teachers were classified 
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TABLE 14 

In-service Training of School Staff 



Percentage Answering Yes* 
1-4 Grade Other Reading Media Resource Spec. Ed. 
Teachers Teachers Teachers Spec. Teachers Toachers 



necBivQQ training 


Q7 


01 


1 f\f\ 


100 


74 


90 


Training helpful 


94 




85 


89 


68 


75 


Training sufficient 


44 


75 


50 


65 


71 


40 
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MAtirifllfl 
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Demonstrations 




69 








65 


Opportunities to 
visit classes 




53 








70 


Extra learning 
time 




89 








75 


Sessions with 
specialist 




51 








65 . 


In-service workshops 




46 








80 


MCPS courses 




41 








40 



♦Blanks indicate the question and response were not Included in the 
instruments for those respondents. In the interviews, the question was 
open-ended and the responses included the categories in the table. 



TABLE 15 
Hours of Training 



Percentage of teachers 

Number of Hours Training Sufficient: Need Additional Help: 
of Training Received Yes No Yes No 



5 or less 


32 


68 


64 


36 


6-10 


46 


54 


50 


50 


10+ 


73 


27 


33 


67 
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according to the achievement level composition of their class, those with 
students in their class who were below grade level in reading were more 
likely to feel that the in-service training they received was 
insufficient and to want additional help (see Table 16). As discussed 
above, instruction to below-grade-level students was less likely to reflect 
the curriculum. Combined with the information on training needs, this 
certainly suggests that the teachers with lower achieving students found the 
curriculum more difficult to implement than those whose students were on or 
above grade level. 

Curriculum Materials 

Aside from in-service training, the major support for schools and teachers 
in the process of implementing the new reading/ language arts curriculum is 
the provision of various support materials for the program. These materials 
include the program overview manual, the instructional guides, the scope and 
sequence charts, the core book lists, and the basal correlation sheets. In 
general, the various materials are available in the schools and are being 
utilized by teachers. When asked, teachers reported that they found them 
helpful. However, many teachers are not satisfied with particular aspects 
of the instructional guides, the core book lists, and the basal correlation 
shee ts . 

The instructional guides provide the major support for the program. These 
include specification of the program's performance objectives, a description 
of the various forms of discourse, and some sample lessons of various types. 
These were only intended to be suggestive of types of instructional units 
teachers could and should develop for the program. The program developers 
intended for teachers to adapt the units to meet the needs of the particular 
students in their class and to develop new units for their own use. 

While teachers generally indicated familiarity with these guides, the level 
of their use may vary more than the data suggest. Teacher comments about 
problems with the curriculum frequently focused on the guides, suggesting 
their importance to the implementation process. Forty-four percent of the 
teachers interviewed mentioned some problems with the use of the guides. 
Poor organization of the guides was the most common concern. There is no 
index by objectives so that the teacher can quickly locate materials and 
units designed to cover oarti^.ular objectives. The lack of an index and the 
continuing piecemeal distribution of new units, other inserts, revisions, 
etc., as well as the fact &At much of the material continued to be labeled 
"draft" which created an impression of carelessness and raised questions in 
the minds of school staff about the priority to be given to program 
implementation. One respondent characterized this process as the "dribble 
approach" to implementation. This combined with print for many portions of 
the guides that virtually all respondents found forbiddingly small did not 
enhance their use. While area staff commented that better packaging would 
help "sell" the program, many also felt that additional units needed to be 
generated in an ongoing fashion so that the system did not have to rely on 
every teacher developing his/her own lessons but could choose from those 
available. Within an area, exemplary lesson units developed by one teacher 
may get passed along informally through the network of area tear-her 
specialists; but there is no formal means for making such units available to 
all teachers. 
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TABLE 16 

Class Composition and Training Needs 



Percentage of Teachers 

Training Sufficient: Need Additional Help: 
Yea No Yes No 



First Grade: 



No below- grade- 
level students 55 45 43 57 



Below-grade-level 

students in class 34 66 59 41 



Fourth Grade: 



All students above 

grade level 100 100 

All students on 

grade level 60 40 20 80 

Students on/above 

grade level 78 22 38 62 

Students include 

below grade level 33 67 59 41 

All students below 

grade level 0 100 100 0 
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Respondents also noted some problems with the core book lists and the basal 
correlation sheets. The core list provides a set of recommended tradebooks 
with different forms of discourse for different reading levels for each 
grade and is supposed to be used in conjunction with the guides by teachers 
in planning instruction. Area staff in the interviews strongly recommended 
the need to continually update and to expand the core book list to improve 
its usefulness. Over 60 percent of the reading teachers and media 
specialists mentioned that the list was out of date or did not include 
enough books for different reading levels or different forms of discourse. 
The basal correlation sheets were designed to show how the program 
objectives can be interfaced with basal reader stories. While most area 
staff felt these were helpful, several supervisors questioned whether the 
sheets only reinforced teachers' use of base Is. The correlation sheets were 
not available for all series nor apparently in all schools; school reading 
teachers must request a copy from area staff. Fifty-nine percent of the 
teachers interviewed were aware of the availability of the sheets, although 
84 percent of the reading teachers indicated they used the sheets. Area 
teacher specialists noted in the interviews that the sheets need to be 
prepared for more basal series and that more copies are needed to distribute 
to schools. 

MANAGING IMPLEMENTATION 

In order for the new curriculum to be implemented in classrooms, both the 
area offices and schools must assume certain responsibilities for managing 
the implementation process. As noted above, DAS can develop and recommend 
the use of curriculum tests, record systems, or other support materials, but 
it is the area associate superintendents who have authority to establish 
requirements. While the study found implementation problems existed across 
areas, even within one area, standards and requirements might vary depending 
on the elementary school supervisor and the teacher-specialist assigned to a 
particular school. Much of the implementation research confirms that 
managing implementation requires "forceful leadership... that sets clear 
expectations of what is to be done by whom with what effect." 5 Data were 
collected during the first year of the study on how the implementation 
process is monitored and the management responsibilities of staff at various 
levels. The study found problems in these areas: 

o Ad hoc and individual criteria for monitoring implementation were 
in use but no overall standards were applied. 

o School reading teachers have a role conflict between providing 
direct instruction versus serving as a resource to teachers and 
assisting principals in implementation. 
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approved; otherwise, it is up to area offices to prepare the sheets. 

9. Loucks-Horsley, op cit. , p. 9. 
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Data from the first year of the study Indicate that neither the criteria for 
monitoring nor who monitors has been clearly defined with respect to tbi new 
curriculum. At the time of the Interviews for the study, what was 
considered evidence of Implementation appeared to be an Individual decision 
for each supervisor and/or teacher apecialist. No overall summary measure 
of the level of Implementation was being used. Several supervisors 
mentioned the need for a checklist or guide Indicating those specific 
features that would reflect Implementation of the new curriculum such as 
teacher use of trade books and other materials, use of Instructional units, 
use of required forms of discourse, and use of planning sheets. Such a 
checklist would be useful to both area staff and principals for monitoring 
purposes. The development of such a checklist was In progress In at least 
one area by the end of the summer of 1983. Because of the area 
administrative structure, there was no efficient means to share materials 
developed In one area with staff In the other areas. 

While area supervisors are formally responsible for monitoring the 
Implementation of the reading curriculum In the schools assigned to them, 
several indicated that they rely heavily on the Information supplied by the 
teacher specialists who serve their pchools. Not all supervisors of 
elementary Instruction are specialists In reading. While all reported 
receiving useful in-service training from Central Office staff and in area 
in-service workshops, several felt the program required a great deal of 
independent review of the materials in order to become thoroughly acquainted 
with the program. In addition, several supervisors felt the need for 
additional training for all supervisors directed to their role in monitoring 
the program and suggested indicators of implementation. 

Area supervisors noted that since many aspects of the curriculum were 
recommended and not required it was difficult to know what features were 
critical to the implementation process. Their lack of clarity was evidently 
communicated to the principals with which they worked. While 94 percent of 
the principals thought the area office monitored implementation, few had any 
idea of what specific indicators area staff used to check the level of 
implementation in their school. This ambiguity about just what was being 
monitored also conveyed a message about the importance of program 
Implementation. Indeed, 50 percent of the principals indicated in the 
interviews that IPR/LA was not the reading program but was added to the 
traditional program or used as a supplement. Several of the area 
respondents felt implementation was not taken seriously precisely' because no 
clear statement of reo u?ed program components was available. "Higher 
levels" must "lay it n the line" that this is the mandated reading program 
and that it will "cot {o h ay." "A firmer commitment is necessary irom the 
Central Office in order f r Implementation to be taken seriously." 

Data from the school scaff Interviews indicate similar confusions over who 
has monitoring responsibilities and what were the indicators of 
implementation within schools. While 89 percent of the principals claimed 
they shared reaponaibility for monitoring implementation with the school 
reading teacher, only 24 percent of the reading teachers included monitoring 
as one of their duties in implementing the program. In addition, while 94 
percent of the principals stated that they reviewed teacher plans as 
indicators of program implementation, only 13 percent of the teachers 



interviewed thought their Instructional plana were used to monitor 
Implementation. 

Staff Roles 

Aside from the problems created by the ambiguity In criteria for monitoring 
lmplementa tlon f the data suggest that the process has also led to some 
concern over the roles of particular staff and changes resulting from 
implementation. Several area staff expressed concern during the Interviews 
about changes in the role of the reading teacher. These respondents felt 
the "old* 9 guidelines for reading teachers no longer fit. Reading teachers 
may now be expected to be part of an Implementation team and to serve as a 
resource/support to teachers, thus leaving less time for direct instruction. 
The problem is even greater when the reading teacher is less than full time, 
as they face the difficult task of responding to curriculum demands as well 
as tutoring students in an even more limited time span. According to 
several area staff, the role of the reading teacher must be re-examined, re- 
defined and updated to reflect the demands of the new reading curriculum. 

As noted above, the reading teachers were less likely to acknowledge a 
monitoring role, perhaps because it did not fit with the accepted definition 
of their role but placed them in an awkward position between the principal 
as supervisor and the teachers as colleagues. At any rate, both classroom 
teachers (77Z) and principals (892) did perceive the reading teachors as 
involved In monitoring. In part, principals may have been reflecting a 
desire to be able to rely on the reading teacher to assume some of these 
Implementation tasks, especially if the principal was unfamiliar with the 
program or uncomfortable as an Instructional leader in reading/language 
arts. Principals who shared monitoring and program coordination with the 
reading teacher were more likely to think the reading teacher's role had 
changed and to see his/her main role more as a resource rather than one that 
provides instruction (see Table 17). 

There is general agreement among respondents that the reading teacher has an 
expanded resource support role. Ninety-four percent of the classroom 
teachers Interviewed said that the rending teacher acted as a resource, 
while 65 percent indicated that the reading teacher provided direct 
instruction to their students. Reading teachers themselves affirmed that 
their allocation of time had changed. Sixty-two percent Indicated that they 
spent less time on direct Instruction now and more on serving as a resource 
as a result of the new curriculum. However, 81 percent of the reading 
teachers still Indicated that direct instruction took up the largest 
proportion of their time, with 62 percent stating that acting as a resource 
was the second most time-consuming activity. Subsequent data collection 
should allow more detailed exploration of the changes occurring In the 
reading teacher's role because of the program. 



IMPLICATIONS 

The problems In implementation of the reading/language arts program have 
implications at two levels: for the curriculum Itself, and for the general 
process of curriculum Implementation and improvement in MCPS. 
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TABLE 17 

Principals' Views of Reading Teacher Role 



Percentage of Principals Reporting That 



Reading teacher 

role changed: Yes No 

Reading teacher monitors: 

Yes 53 47 

No 0 100 



Reading teacher's main role is: 
Direct Instruction Resource 

Reading teacher coordinate?: 

Yes 17 S3 

No 58 42 
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IPR/LA Implementation 

First, with respect to IPR/LA Implementation Itself, a number of 
Implications can be drawn from the specific study findings. These concern 
Instructional practice, use of tests, ln-servlce training and support 
matt rials, and program monitoring. 

Some, but not all, aspects of Instructional practice matched the Board- 
mandated Program of Studies. Continued effort Is needed In order to 
Increase the use of these practices. The Issue Is of particular concern for 
low-achieving students. Teachers have not been convinced that the program 
Is appropriate for low achievers. The Instruction provided to these 
students was less likely to match program standards than that provided to 
higher achievers. Since the curriculum was mandated by the Board for all 
students, a greater effort needs to be made to provide adaptations which 
better meet these students' needs, to convince teachers or the efficacy of 
the program In Its present form with low achievers, or to Improve management 
procedures. 

The purpose of the curriculum tests and the procedures for their use are not 
well understood. These ambiguities as well as the lack of a record-keeping 
system have resulted In a mixed message about the need to assess performance 
on curriculum objectives. These problems will be remedied In large part by 
the Inclusion of these tests In the mandated annual testing program 
Initiated this spring and the provision of a record-keeping system for use 
with the program, which Is expected to be available in 1984-85. Continued 
efforts need to be directed, however, toward helping schools understand how 
to use the data produced by the tests. 

Neither <n-servlce training nor support materials have been adequate. While 
a specific number of additional hours cannot -•. recommended, more and/or 
Improved training Is clearly needed, especially on how to adapt and use the 
program with low achievers. In addition, teachers want a better 
understanding of program principles and more practical demonstration of 
classroom procedures using the new program. Training should Include 
assistance to teachers In adapting and developing new lessons based on the 
program objectives. Support materials need to be Improved to Increase their 
usefulness: Instructional guides should be Indexed by objectives, and core- 
book lists and basal correlation sheets should be updated and expanded. 

A clear statement of what should happen In the classroom Is also necessary 
for effective monitoring of program Implementation. Principals and teachers 
need a list of specific program components (type of plans required, 
appropriate Instructional practices, use of tradebooks) to use In assessing 
the progress of Implementation. However, unless this results In a statement 
of required program components, Implementation may continue to be piecemeal. 

Curriculum Implementation In MCPS 

Problems such as those mentioned above are not unique to Montgomery County, 
and time alone does not provide a cure. Study after study has documented 
Innovative Ideas that were developed but never Implemented In any but a 
superficial way. It has become clear that program Implementation Is, under 
the best of circumstances, difficult and must be carefully orchestrated If 
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It la to succeed. It It time to take a very serious look at how MCPS goes 
about the program Implementation task, as this Is only one of a series of 
studies In which Implementation problems have been found. DEA studies In 
other program areas (the Instructional System In Mathematics, career 
education, gifted and talented) have documented similar problems In the 
Implementation process. In MCPS, curriculum development, monitoring, and 
Instruction have been separated, and there are no systematic procedures In 
place for curriculum Implementation that tie the various elements together. 
Colleglal monitoring and good will are not sufficient to accomplish 
Implementation. Without systematic procedures which both support and 
enforce Implementation, the result Is Board-approved programs that are never 
fully Implemented. 

Program Implementation Is a complex process. Table 18 provides an example 
of the various stages/activities In the Implementation process as Identified 
in recent research (Loucks-Horsley, 1984). While it is certainly not 
necessary to follow each step to the letter, the model Is itself Informative 
and contrasts sharply with practice typical of MCPS. MCPS has a number of 
curriculum reform efforts In various stages of dissemination In the schools. 
In the English Language Arts, the writing and speaking program Is currently 
being piloted, 10 and plans call for broader Implementation next year. 
Before such expansion Is scheduled, DEA feels It is critical that a 
complete, multlyear plan for Implementation be prepared. Rather than 
continue with- the "dribble" approach to program Implementation that has been 
characteristic to date, It is time to take a more systematic approach to the 
problem. Without systematic procedures for Implementation that specify 
responsibilities, establish requirements, and provide resources, there is no 
-reason to expect that the writing program will fare any better than those 
examined to date. 



10. Approved by the Board of Education on May 21, 1984. 
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TABLE 18 
Implementation Activities 



Initiation Phase 

Assessing needs , strengths, and resources 
Assessing current practice 

Setting clear goals, objectives, and expectations 
Selecting or developing a new practice 
Creating awareness 

Assigning roles and responsibilities 
Establishing commitment 
Developing game plans 
Allocating resources 
Providing materials 
Arranging training 

Making schedule and organizational changes in school 
Helping teachers plan implementation 

Implementation Phase 

Initial training 

Problem-solving and trouble-shooting 
Providing follow-up training 
Monitoring classrooms for use 
Evaluating implement a tior outcomes 
Evaluating ultimate outcomes 

Institutionalization Phase 

Training new or reassigned staff 
Conducting follow-up and refresher sessions 
Incorporating program into curriculum guidelines 
Routinely purchasing new materials and supplies 
Establishing a budget-line item 



Source: Loucks-Horsley, c p> clt , p. 14 
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DATA COLLECTION PROCEDURES AND INSTRUMENTS 



Reading study staff collected information from sample schools and area 
personnel during the first year of the study (1982-83). In the fall, 
interviews were conducted with principals, first- and fourth-grade teachers, 
and area staff. The interview guides were designed to elicit parallel 
information from the different administrative and teaching staff involved in 
implementing the program. The topics covered included planning, in-service 
training, the role of school-based specialists in implementation, the use of 
curriculum support materials, instructional practices in reading, and 
changes due to the curriculum, attitudes towards the program, and its 
appropriateness for sulents stall achievement levels. While question 
responses were close-ended where possible based on pilot study information, 
many questions in the interview guides were open-ended to allow respondents 
to elaborate on their perceptions of the program and its impact. Separate 
interview guides were developed for each of the respondents: area 
supervisors of elementary instruction, area teacher specialists, principals, 
and the classroom teachers. The interviews each took approximately 45 
minutes to one hour. All respondents were provided with a copy of the 
interview guide in advance of the interview. In addition, to minimize 
teacher burden, a substitute was provided for each teacher's class while the 
teacher was being interviewed. 

Supplementary questionnaires were given to the rest of the classroom 
teachers in the sample schools as well as to school specialists in February. 
These ins rumen ts covered the same topics but in less depth, using a fixed 
response ormat. Separate questionnaire forms were developed for each of 
the d'~.«rent respondents: classroom teachers, school reading teachers, 
media specialists, special education resource room teachers, and special 
education teachers. Questionnaires were estimated to take approximately 30 
minutes to complete by the respondent. The forms were sent to participating 
school principals to distribute to staff, collect, and return to DEA. Table 
A-l lists the instruments, number of respondents, and response rates. 
Although the questions were similarly phrased, the differences in format, 
procedure, and timing (February vs. November) necessitate separate reporting 
of responses from the interview and questionnaire data. 

These forms were used in tho eighteen schools in the sample that had begun 
to implement the program. In the two nonimplementing schools, brief 
interviews were conducted with principals and a short questionnaire was 
distributed to all classroom teachers. These instruments omitted the 
questions about program implementation and only asked fot som<? baseline 
information on reading instruction practices. Since both of the 
nonimplementing schools in the sample had experienced some consolidation, a 
number of the classroom teachers actually had had experience implementing 
the program in their previous schools. Classroom practice information 
collected from these schools could thus not be interpreted v.s evidence of 
reading instructional practices prior to implementation of I PR /LA. 

In order to get a clearer picture of instructional practices, classroom 
observations were also conducted in sample classrooms. Reading instruction 
was observed during three time periods (fall, winter, and spring) in the 
first- and fourth-grade classrooms in the study. Two separate observation 
procedures were used in order to minimize teacher burden and yet capture 
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TABLE A-l 

Data Collection Instruments and Respondents 



Ins t rumen ts 


NUUUB17 


Response Kace 
(Percentage) 


Supervisor of Elementary Instruction Interview 
Guide 


0 




Area Teacher Soeclallst Interview Guide 


o 


100 


Elementary School Principal Interview Guide 


18 


100 


Elementary Classroom Teacher Interview Guide 


71 


100 


Elementary Classroom Teacher Questionnaire 


184 


93 


School* based Reading Teacher Questionnaire 


21 


100 


Media Specialist Questionnaire 


18 


100 


School-based Special Education Resource Room 
Teacher Questionnaire 


20 


• 

95 


Elementary Special Education Teacher 
Questionnaire 


21 


95 
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both teaching behaviors and student instruction received: an "in-depth" and 
a "snapshot" observation. The in-depth instrument was designed for group/ 
whole class observations and was used to code teaching behaviors for an 
entire reading period while instruction was taking place under teacher 
direction. The shapshot instrument was designed to observe individual 
students and was used to code student behaviors for s five-minute period 
regardless of whether the student was under tescher direction. A copy of 
each instrument snd the definitions of terms used in coding is included at 
the end of this sppendix. Although it wss stressed that the observations 
were intended to assist in program evaluation and not assessment of the 
quality of individual teaching, the procedures caused considerable anxiety. 

The in-depth instruments was used in one first and one fourth-grade 
classroom in each of the sample schools three times during the school year. 
The observer remsined in the classroom for the entire reading period, 
"joining" all reading groups that the teacher instructed during that period. 
The actual time spent in each classroom varied depending on the length of 
time individual teachers allocated for reading instruction. The Instrument 
used by the observer lists nineteen different teschlng behaviors/ strategies 
divided into the following six areas: Vocabulary Development, 
Comprehension, Decoding/Structural Analysis, Materials, Instructional 
Practices/Management, and Interest and Motivation. These were selected to 
reflect aspects of instruction likely to be affected by implementation of 
the new curriculum. An open-ended section at the end of the instrument was 
provided for comments relating to classroom displays and grouping practices. 
The observer coded whether each behavior had occurred or not separately for 
each of the reading groups in the class. There are no timing procedures 
associated with this instrument. 

The alternative "snapshot" procedure was used in the other first- and 
fourth-grade classrooms in the study. In these classes, a students was 
randomly selected from each of the reading groups to represent students in 
that group. These target students were observed for five minutes three 
times during each of the observation periods, nine times during the entire 
year. The number of students per classroom varied depending on the number 
of reading groups and ranged from one to six. The observer remained in the 
classroom for part of the reading period following individuals students. 
The instrument used for this observation procedure contains nineteen 
behaviors grouped into four observation categories for which individual 
students were to be coded: Task-attending Behaviors, Work Setting, Reading 
Instructional Task, snd Materials. These were selected as descriptive of 
the basic elements of reading instruction regardless of curriculum. As 
noted above, this was a timed observation. The observer coded each target 
student in the areas mentioned once every 15 seconds for a period of five 
minutes (20 observations). 

Although the observations were unannounced within the specified time periods 
in the fall, winter, and spring, the observers needed to know the individusl 
teachers' schedules for instruction to plan observation visits. In 
addition, the number of reading groups and the students in each had to be 
determined in order to select students for the "snapshots." Teachers were 
therefore asked to fill out forms providing information on their reading 
instruction schedule for each of the six-week periods as well as a roster 
listing the students in their class by reading group. Teachers were asked 
to number the groups in order by reading achievement levrl, so that this 
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information could be used to group the observation data. Since the top 
group In one class could be at s different level from the top group In 
snother class, at the end of the year, supplementary Information on the 
reading level and grade level (on, above, or below) of each group was 
obtained from teachers. It should be noted that observations were limited 
to students receiving regular reading Instruction In the classroom; pullout 
Instruction for whatever purpose (remedial, gifted and talented, ESOL, etc.) 
was not observed, and the observation data, therefore, cannot be used to 
characterize such Instruction. Obtaining accurate schedules for reading 
instruction for a six-week period in advance proved quite difficult. An 
unanticipated study observation has been the complexity of elementary school 
schedules and the extent to which Instruction Is continuously Interrupted 
for various special events—field trips, assemblies, swimming, parties for 
every conceivable holiday. 

Seven observers were selected to conduct the observations for the Reading 
Study. All had extensive classroom experience and either a Master's degree 
In reading or special training and experience as a classroom observer. The 
seven observers each were responsible for observing classrooms in two or 
three schools. The observers participated in a training session in October 
1982 to prepare for the observations. During the workshop, the study design 
was reviewed, th? curriculum guides for grades one and four were examined, 
the observation Instruments were reviewed and explained In detail, and 
assignments were made. Individual observers were responsible for developing 
their own schedule for conducting the observations. With the exception of 
one teacher who was excused from the observations because of personal 
problems, all the selected classes were observed at least once during each 
time period; only a few observations were not completed because of 
scheduling difficulties. 

The observation instruments were Intended to describe instruction and 
minimize the extent to which observer judgment was needed. Nevertheless, 
some judgment was obviously necessary. To assure quality control for the 
study, frequent contact was maintained with the observers, and special 
procedures were used to ensure that interrater reliability remained high. 
In order to check interrater reliability, the Reading Study observation 
supervisor joined each observer for each type of observation in November, 
February, and May. These joint observations were conducted in all 20 
schools in the study. Both the supervisor and the observer coded 
simultaneously, and then percent agreement was calculated using the 
supervisor's coding as the standard. For the in-depth instrument, the 
average reliability across the observers was 90 percent in November, 88 
percent in February, and 89 percent in May. For the snspshot instrument, 
the average reliability was only slightly less: 86 percent in November, 87 
percent in February, and 87 percent in May. 

This joint coding procedure allowed the supervisor and the observers to 
pinpoint coding problems as they occurred and making corrections as they 
went along. In general, the in-depth procedure presented fewer difficulties 
than the snapshot procedure. Since the in-depth coding was not timed, the 
observers had ample opportunity to reflect on teaching throughout the entire 
reading period rather than brief moments with individual students. 
Observers also found it easier to join reading groups working under the 
teacher's direction in one area of the room thsn to observe target students 
working in a variety of settings tn many areas throughout the room. In 
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addition, because of transitions, the snapshot observations took more than 
five minutes of elapsed time to record five minutes of instruction. Thus, 
teachers observed with this procedure had an observer in their classroom for 
a longer period of time than anticipated. Nevertheless, the reliability for 
both Instruments was about the same. For both instruments, where the 
reliability checks or the feedback meetings with observers indicated 
ambiguities or problems in interpreting the data, these observation 
categories were not included in the report. 

Student data were* also collected in first and fourth grades. To provide 
baseline information on global reading skills, an individually administered 
reading test (the Woodcock Reading Mastery Test) was given to first graders 
in the fall, and fourth graders took the California Achievement Test 
(Reading Subtests). Testers hired by DEA or the school reading teacher gave 
the Woodcock test, and student data were left with the schools to use as 
they wished. Scores on the criterion-referenced tests in narration 
administered by schools in October and January were supplied to DEA by DAS. 
As part of the study, fourth-grade teachers were also asked in the fall to 
administer a brief (16 items) Inventory of reading attitudes to their 
students. Answer sheets were submitted to DEA and school-level summary 
scores were later returned to the schools. The inventory was not felt to be 
appropriate for first graders. 

To assess some additional reading skills not included on the curriculum 
tests and to provide an end-or-year measure of student performance, DEA 
developed a separate criterion-referenced test for first and fourth grade. 
To ensure uniformity of administration of these new tests and to ease the 
burden on the schools, DEA staff went to the schools to administer the 
tests. Scoring took place over the summer, and the schools received reports 
on the 'Its in September, 1983. Additional data on student attitudes 
were .oUe from a subsample of first- and fourth-grade parents In the 
study . "hr^is through a telephone survey. The student data collected in the 
first year is Intended primarily as baseline information to be used in 
conjunction with successive years' data to measure student growth and relate 
gains to program Implementation. These data have been returned to the 
schools for their use and will be used In subsequent analyses, but they are 
not included in the first-year report. 
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In-depth Instrument 



School 



Date 



Teacher. 
Grade 



Number of Reading Groups on Roster 
(If M 0 M please explain) 



Observer/Rater 



Start/Stop Time Observed 



TEACHING BEHAVIORS OBSERVED DURING READING PERIOD 

DIRECTIONS: Consider today's classroom experience. Concentrate on the 
teaching behaviors observed during the reading period. Indicate whether you 
observed the particular behaviors listed by checking the appropriate box next 
to each statement for each group observed. Please put a zero when behr.^ior is 
not observed. If chere is whole class instruction for the reading period, 
code behaviors observed in column 1 of observational form and note that on 
this form. Please note comment section at the end of the observation 
instrument. 



I. VOCABULARY DEVELOPMENT 



Reading Groups 
) 2 3 i 



a. Teacher presents and develops selected words 

for reading and/or listening lesson in meaningful 
context. 



b. Teacher presents selected words visually. 



c. Teacher encourages student use of selected 
words in class discussions. 
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II. COMPREHKNSION 

Reading Group 
1 2 3 

a. ' Teacher clearly establishes purpose for reading, 

and/or listening, (i.e., read this selection to 
find out, listen for rhyming words in 
selection*) 

b. Teacher uses open-ended questioning strategies 
that promote critical thinking and discussion, 
(i.e. "How", and "Why" questions) 

Specify some examples of questions: 



c. Teacher encourages students to refer back to 
selection to prove/justify answers, (i.e., 
find the sentence which gave you the clue) 

d. Teacher encourages students to predict while 
reading a selection, (i.e., "What do you think 
will happen next? Why do you think . . . .?") 

e. Teacher relates reading text to student's prior 
background knowledge and experience, (i.e., "Did 
you ever ...?", "Has anyone ever been to ...?," or 
"How would you feel if it happened to you?") 
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III. DECODING/STRUCTURAL ANALYS IS 



Reading Groups 
12 3 4 

a. Teacher uses isolated phonic 

drills to help students decode 

unfani.Uar words (i.e. letter/sound 
relationships taught in isolation- 
no context) 

b. Teacher use context clues to help * ' 

students decode unfamiliar words. 

(i.e., letter/sound relationship 
taught using context sentences » 
paragraphs , whole stories) 
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IV. MATERIALS 



Reading Groups 



1 



a. Teacher uses basal readers during the reading 
period . 

b. Teacher uses tradobooks during the 

reading period. 



Specify book(s) : 



c. Teacher uses content textbooks during the reading 
period 



Specify book(s): 



d. Teacher uses materials other than basal readers 
textbooks , tradebooks during the reading period, 
(i.e., language experience stories, ditto sheets) 



Specify materials: 
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INSTRUCTIONAL PRACTICES /MANAGEMENT 



a. Teacher uses "wait time" strategy (I.e., teache 
allows at least five seconds between question 
and responses and/or between responses and 
asking next question.) 

b. Teacher assumes a central/active role in 
providing reading instruction, (i.e.* teacher 
directed instruction) 

c. Teacher involves more than half the students in 
discussion and encourages active participation 
of students. 

d. Teacher uses enrichment /extending activities 
related to reading/langu u e arts to provide for 
creative interpretation (i.e. dramatize story, 
make poster of favorite character's adventure, 
rewrite story with new ending, construct mural 
or model of. • •) 

Specify activities: 
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VI. INTEREST ANU MOTIVATION 



Reading Groups 



1 



2 3 



4 



a. Teacher creates an interest and motivation for 

the reading lesson. 

COMMENTS : Reflect on the classroom experience immediately following the 
observational period. Please comment briefly on each of the 
following areas as they relate to today's reading period. 

In area a., "Attentiveness of groups during instruction," please 
note whether students, in general, were on- task in their reading 
group. On- task behaviors include such behaviors as: listening 
attentively to the teacher, responding to questions, asking 
questions, etc. Off-task behaviors include such behaviors as: 
gazing out the window, nudging a peer, whispering to a peer, etc. 

In area b. , "Language instruction integrated with content 
subjects," please note whether the teacher provided any reading 
skill instruction to help students read and understand social 
studies, science, math, etc. materials during the reading period. 

In area c, "Graphic displays and/or verbal expressions related to 
the MCFS Reading/Language Arts Program," please note whether IPR/LA 
objectives, vocabulary terms (i.e. , discourse, narration, 
exposition) , materials (i.e. , multiple copies of tradebooks) , 
charts or diagrams (i.e., webbing, story structure-character, plot 
setting) are evident in classroom displays or whether the teacher 
uses concepts/terms from the curriculum (i.e. character trait, 
setting, conflict, resolution). 

In area d., "Changes in the instru* "ional groupings," please note 
whether there was any variation/flexibility in instructional 
groupings during the reading period (i.e., whole class, small 
groups, individuals, heterogeneous and homogeneous groups). 

In area e., "Variation in length/type of reading materials for 
different ability groups, please note whether there was variation 
in the amount of content covered (number of pages) and the types of 
materials used for the varying ability groups. 

In area f., "Reading skills covered during instruction," please 
note specific areas of instruction (i.e., vocabulary development, 
comprehension, decoding, etc.) covered. curing the reading periods, 
regardless of whether specific behaviors were observed in those 
areas. 

a. Attentiveness of groups during instruction 
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b. Language Instruction integrated with content subjects 



c. Graphic displays and/or verbal expressions related to the MCPS 
Reading/Language Arts Program 



d. Changes in the instructional groupings 



e. Variation in length/type of reading materials for different 
. ability groups 



f • Reading skills covered during instruction 
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Snapshot Instrument 
OBSERVATIONAL CHECKLIST FOR TARGET STUDENTS 
INSTRUCTIONS TO OBSERVERS: 

Each target student is observed for one 5-minute period during the "snap-shot" visit 
in tho classroom. Check the appropriate column on the observational checklist form 
•very 15 seconds for five minutes (observation numbers 1-20) for each student until 
all target students have been observed for the five minute period. 

A check is made in column 1 to indicate whether the student is attending to the task 
at hand. On task-behaviors include such behaviors as: attending to words on the 
board, listening attent'vely to the teacher, responding to questions, asking 
questions, reading aloud, and reading silently. Off- task behaviors include such 
behaviors as: gazing out the window, nudging a peer, whispering to a peer, and 
getting up to sharpen a pencil. Cite example(s) of the off-t.vdk behavior(s) 
observed qn the observational form. 

i 

A check is made in column 2 to indicate whether the student is working under teacher 
directions, working independently, working with another adult or student (s) in the 
classroom. The category "working under teacher directions" is used in all instances 
where the student is under teacher guidance. This category includes total 
classroom, small group, or individualized instruction. The category "working 
independently" is used to designate time when the student is working independently 
and not under the direction of the teacher. In the category "with other adult", 
please specify the adult (parent, aide, etc.) where possible, on observational form. 

A check is made in column three to indicate whether the reading instructional task 
focused on: a) isolated letters or words or both, b) reading context c) non- 
reading activities such as listening, speaking and writing. Students can be 
involved in work focusing on letters , words, and contextual reading using basal 
readers, vorkbooks, dittos, etc. If student is engaged in reading context please 
specify on observation form, whether students are: a) reading 

words in the context of sentences , b) reading words in the context of paragraphs , or 
c) reading words in the context of whole st ories . Reading- related activities are 
those activities congruent with "reading instruction" but not actually involving 
pupils in reading, for example, answering a question, waving one's hand to answer a 
question, asking a question, and writing a response. If student is engaged in task 
not related to reading (i.e., art project, math homework, etc.) please record in 
other column and specify the task on observational form. (Where student is 
engaged in two activities and you cannot select one, use a 1/2 to indicate primary, 
secondary emphasis.) 7 

A check is made in column A to indicate the materials, used by the target student 
during each observational period. If student is reading a tradebook please specify 
the book, where possible, on observational form. 

Note your impressions regarding teacher expectations of target student success. 
Based on the interactions (verbal and nonverbal) among the teaching adults and 
target students during the "snap-shot" visit indicate under "IMPRESSIONS", on the 
observational form, whether the teacher showed negative expectations, positive 
expectations or no overall expectations for the academic success of the target 
stuc^s (i.e., positive expectations-teacher shows confidence in cognitive 
abili of student, predicts that student will be able to accomplish tasks, etc.) 
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School _ 
Teec her 
Grede 



OISERVATIONAL CHECKLIST FOE TARGET STUDENTS 



hut •* ^ ^ _ r _ i 
OMorvcr/Ratar 
Terget Student 



OISERVATION 
NUHBFR 



I. TASK ATTENDING 
■EHAVlOR 



2. WORKING. 



3. HEAPING INSTRUCTIONAL 



ASK 



MATERIALS USED 



On-Teek 



♦Off-Teek 



Undtr 
Teacher 
Direct lone 



Independ- 
dently 



♦With 
Other 
Adult 



With 
Othtr 
Student (a) 



Letters 
and/or 
Word* 



* Reading 

In 
Context 



Reading-Related 
Activities 



Lleten 
Int 



Writ- 
ing 



Speak 



♦♦Other 



Ditttl 
Reader 



♦Treda- 

book 



Uurk 
book 
Work 
•heet 



Other 



♦Specify off-teek 
behavlor(a) 



♦Specify other edult 



♦Specify 



♦Specify 



♦Specify Tmdebook 



"Specify 



IMPRESSIONS J 



ERIC 



BEST COPY AVA I LABL E 



61 



